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A human-in-the-loop experiment was conducted at the NASA Ames and Langley Research Centers, investigating 
the En Route Free Maneuvering component of a future air traffic management concept termed Distributed 
Air/Ground Traffic Management (DAG-TM). NASA Langley test subject pilots used the Autonomous Operations 
Planner (AOP) airborne toolset to detect and resolve traffic conflicts, interacting with subject pilots and air traffic 
controllers at NASA Ames. Experimental results are presented, focusing on conflict resolution maneuver choices, 
AOP resolution guidance acceptability, and performance metrics. Based on these results, suggestions are made to 
further improve the AOP interface and functionality. 


Introduction 

In today’s air transportation business environment, 
aircraft operators are increasingly looking for means 
to increase flight efficiency. However, with air travel 
demand once again rising to levels that exacerbate 
delays and challenge the capacity of the National 
Airspace System (FAA, 2004), large efficiency 
improvements may be difficult to realize under 
current operational conditions. As a result, it has been 
acknowledged that a transformational, rather than 
evolutionary, approach to air traffic management 
modernization is needed (DOT, 2004). 

As part of the Advanced Air Transportation 
Technologies project, NASA has developed such a 
far-term, transformational concept, called Distributed 
Air/Ground Traffic Management (DAG-TM) 
(NASA, 1999). The goals of DAG-TM are to 
increase efficiency and maintain safety through a 
redistribution of decision-making authority among 
airborne and ground-based elements of the air 
transportation system. It is a gate-to-gate concept, 
addressing all flight phases from dispatch to arrival. 

En Route Free Maneuvering 

En Route Free Maneuvering is one component of 
DAG-TM, addressing the en route and terminal- 
transition phases of flight. In an En Route Free 
Maneuvering environment, trained crews of equipped 
aircraft assume responsibility for traffic separation. 
Such crews would be free to modify their flight path 
in real time, without approval from an air traffic 
controller, as long as basic flow management 
initiatives are complied with (e.g., crossing a terminal 
airspace entry point at a specified time). These flights 
would operate under a new set of flight rules called 
Autonomous Flight Rules (AFR). 


Except for busy terminal areas, where AFR 
operations would not be permitted, AFR traffic 
would be integrated with Instrument Flight Rules 
(IFR) traffic. AFR flight crews would be responsible 
for separation from both IFR and other AFR aircraft. 
Air traffic controllers would issue flow management 
constraints to all aircraft, and continue to provide 
separation among IFR aircraft, accommodating those 
operators who choose not to equip for AFR. By 
distributing separation assurance among multiple 
airborne and ground-based elements in this way, the 
National Airspace System may be able to absorb a 
higher increase in demand beyond what is possible 
with a centralized, ground-based approach. 

Background 

Previous Research 

The work presented in this paper builds upon 
previous studies conducted at NASA as well as initial 
Free Flight research by organizations such as NLR in 
the Netherlands (Hoekstra et al., 2000). Past NASA 
experiments investigated such topics as AFR 
operations in confined airspace and the use of aircraft 
intent for decision making (Krishnamurthy et al., 
2003). 

The Autonomous Operations Planner 
Central to AFR operations are the capabilities of 
airborne conflict prevention, detection, and 
resolution, as well as adherence to traffic flow 
management constraints. It is assumed that pilots 
cannot safely perform these functions without some 
form of decision support. As such, NASA Langley 
Research Center has developed a prototype airborne 
toolset called the Autonomous Operations Planner 
(AOP) (Barhydt & Krishnamurthy, 2004). 



The prototype AOP interface is designed around a 
modern “glass cockpit” flight deck. It provides 
conflict alerts and resolution guidance via the 
navigation display, using state and intent data from 
the ownship and proximate traffic. To meet flow 
constraints, it also generates conflict-free paths that 
achieve Required Times of Arrival (RTAs) at 
waypoints. The AOP has been developed using a 
human-centered approach, with resolution guidance 
complementing the pilot’s choice of control mode. 
For example, when the aircraft is being flown in a 
tactical mode (e.g., a selected heading or altitude) or 
when very near-term conflicts exist, resolution 
guidance is presented as a simple heading or vertical 
speed command. When the aircraft is flown in a 
strategic mode (i.e., coupled to the aircraft’s flight 
management system (FMS)), resolution guidance is 
presented as an FMS route modification. 

Conflicts are displayed by highlighting the intruder 
aircraft and indicating the region of conflict along the 
active flight path with a colored “dog bone.” The 
AOP also provides information to help pilots avoid 
inadvertently creating new conflicts while 
maneuvering. These conflict prevention tools take on 
two forms: Maneuver Restriction Bands and 

Provisional Conflict Alerts. Maneuver Restriction 
Bands are displayed as “no fly” heading and vertical 
speed ranges. Using a “dashed dog bone” symbology, 
Provisional Conflict Alerts show regions of conflict 
along proposed flight paths (e.g., a modified but 
unexecuted FMS route or a selected but unengaged 
heading). Figure 1 shows an example of AOP 
symbology on a Boeing 777-style navigation display. 


Figure 1. AOP Interface with Strategic Resolution 


Experimental Approach 

In summer 2004, the NASA Ames and Langley 
Research Centers jointly conducted a human-in- the- 
loop simulation of En Route Free Maneuvering 
operations (Barhydt & Kopardekar, 2005). This 
experiment extended the previous research in several 
ways. A realistic, mixed AFR-IFR operating 
environment was simulated, including overflight 
aircraft as well as arrivals. The AOP was enhanced to 
provide vertical resolution guidance in addition to 
lateral guidance. In addition, interactions with 
ground-based air traffic controllers were studied. 

This paper presents a subset of the En Route Free 
Maneuvering experimental results, focusing on 
conflict resolution maneuver choices, pilot-reported 
acceptability of AOP guidance, and performance 
metrics, including how pilot compliance with AOP 
affected resolution performance. 

Participants 

Test subjects included 12 pilots at NASA Langley as 
well as pilots and air traffic controllers at NASA 
Ames. The NASA Langley subject pilots were all 
Airline Transport Pilot rated with experience in 
Boeing glass cockpit aircraft. These pilots flew 
workstation-based flight simulators that emulated the 
displays of an AOP-equipped Boeing 777. Additional 
AFR and IFR background traffic was supplied with 
pseudo-pilot stations staffed by research personnel. 

Figure 2 shows the experimental airspace. It 
consisted of simulated high- and low-altitude sectors 
of a portion of Fort Worth Center. The sectors were 
staffed at NASA Ames by five FAA-qualified air 
traffic controllers. They provided separation services 
between IFR aircraft and were given automated tools 
for conflict detection and resolution. In addition, 
researchers acted as pseudo-controllers in large 
“ghost” sectors surrounding the experimental sectors, 
providing limited services to flights entering and 
exiting the subject-controlled airspace. 
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Scenario Design 

The experiment was designed in a within- subjects 
format, with 16 different scenarios. Four different 
traffic conditions were simulated, which varied the 
amount of traffic as well as the relative proportions of 
AFR and IFR overflight aircraft. Table 1 details the 
four traffic conditions. 


Table 1. Traffic Conditions Tested 


Condition 

Avg. Traffic Density 

% IFR 

% AFR 

Cl 

slightly above current 
Monitor Alert parameter 

100% 

0% 

C2 

equal to Cl density 

75% 

25% 

C3 

*1.5 x Cl density 

50% 

50% 

C4 

~2 x Cl density 

35% 

65% 


At each of the four traffic conditions, pilots flew two 
overflight profiles and two arrival profiles. Except for 
Cl scenarios (in which all flights were IFR), subject 
pilots were responsible for resolving scripted and 
unscripted traffic conflicts. AOP alerted pilots to 
conflicts up to 10 minutes prior to predicted Loss of 
Separation (LOS). Pilots were trained to use AOP 
strategic resolution guidance, tactical resolution 
guidance, and (in the case of manual maneuvers) 
conflict prevention information as appropriate to the 
situation. They were also instructed to operate the 
aircraft as they would during line operations. 
Although hand-flying was not available, pilots were 
allowed to use any desired autopilot modes, including 
both FMS-coupled modes and tactical modes. 

Results & Discussion 

The NASA Langley subject pilots encountered a total 
of 500 traffic conflicts throughout the 12 AFR 
scenarios (C2, C3, and C4). For 332 of these 
conflicts, the subject pilot performed a resolution 
maneuver. The analyses presented below show 
results for these conflicts, without distinguishing 
between traffic conditions. The effects of traffic 
density on resolution performance are treated in a 
separate publication (Doble, Barhydt, & Hitt, 2005). 

AOP Compliance 

To examine the effects of AOP resolution maneuver 
compliance on resolution performance, resolution 
maneuvers were divided into six categories, based 
upon whether the maneuver was strategic or tactical 
and whether or not the pilot followed AOP guidance. 
These categories are summarized in Table 2. Two 
different performance metrics were then used to 
evaluate the maneuvers: induced conflicts and 
conflicts requiring multiple resolution maneuvers. 


Table 2. Resolution Compliance Categories 


Category 

Description 

Count 

Strategic 

Comply 

Pilot implements AOP-recommended route 
modification without modifications 

141 

Strategic 

Noncomply 

Pilot edits waypoints before implementing 
AOP-recommended route modification 

0 

Strategic 

Manual 

Pilot ignores or does not seek AOP 
resolution, and manually edits waypoints, 
altitudes, etc. of FMS active route 

15 

Tactical 

Comply 

Pilot maneuvers in direction of AOP- 
recommended heading or vertical speed 

118 

Tactical 

Noncomply 

Pilot maneuvers away from AOP- 
recommended heading or vertical speed 

15 

Tactical 

Manual 

AOP tactical guidance not available, pilot 
implements own lateral or vertical 
maneuver via autopilot mode control panel 

43 


Induced Conflicts. The frequency of induced conflicts 
is a measure of the ability of pilots and AOP to 
account for aircraft other than the intruder when 
calculating a resolution maneuver. An induced 
conflict was defined as a new conflict arising within 
one minute of a previous resolution maneuver and 
directly caused by that maneuver. Figure 3 shows the 
percentage of resolutions inducing a conflict in each 
of the six compliance categories. Results from tests 
indicate no significant differences in the frequency of 
induced conflicts across the three tactical categories 
(X 2 (2, N = 176) = 0.27, p > 0.05), but a significantly 
higher frequency of induced conflicts for Strategic 
Manual maneuvers vs. Strategic Comply maneuvers 
(X 2 (l,Af= 156) = 32.2, p< 0.05). 
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Figure 3. Induced Conflicts vs. AOP Compliance 

The lowest induced conflict rates occurred when 
pilots followed AOP guidance. This highlights the 
advantage of decision support when resolving 
conflicts involving multiple proximate aircraft. It is 
conjectured that the relatively high rate of induced 
conflicts among tactical maneuvers was due 
primarily to two factors: the time to predicted LOS 
when the maneuvers were executed, and the 
characteristics of the AOP tactical resolution 
algorithm. During the experiment, tactical resolution 






maneuvers were generally initiated closer to 
predicted LOS than strategic maneuvers. In such 
situations, especially in the high-density airspace 
simulated in this experiment, some induced conflicts 
may be inherently unavoidable, as the first priority is 
usually to resolve the most critical conflict in a timely 
manner. In addition, for very near-term conflicts 
(under 2 minutes to LOS), the AOP tactical 
resolution algorithm did not take other aircraft into 
account when calculating resolution guidance. This 
algorithm was chosen for its ability to successfully 
resolve complicated conflict situations without the 
need for maneuver coordination between aircraft 
(Eby, 1994). Ongoing research will investigate the 
integration of this algorithm with the AOP conflict 
prevention tools in order to further reduce induced 
conflicts. 

While the significant increase in induced conflicts for 
Strategic Manual resolutions is cause for concern, it 
should be noted that three of these five induced 
conflicts were caused by the same pilot during the 
same scenario. Nevertheless, pilot training and the 
AOP conflict prevention symbology may warrant 
further attention as these subject pilots all 
implemented route modifications despite being 
shown Provisional Conflict Alerts. 

Multiple Resolutions. The frequency of multiple 
resolutions is a measure of the ability of pilots and 
AOP to resolve a conflict and remain out of conflict. 
If a subject pilot was in conflict with the same 
intruder multiple times and implemented more than 
one resolution maneuver, this was noted as a multiple 
resolution conflict. Figure 4 shows the percentage of 
conflicts requiring multiple resolutions in each 
compliance category. Results from x 2 tests indicate 
no significant differences in the frequency of multiple 
resolutions across the strategic categories (x 2 (l, N = 
156) = 1.67, p > 0.05). The differences among 
tactical categories were significant (x 2 (2, N = 176) = 
6.04 ,p < 0.05). 
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Figure 4. Multiple Resolutions vs. AOP Compliance 


The lower multiple resolution rate for maneuvers that 
complied with AOP guidance (vs. manual 
maneuvers) shows the benefits of decision support 
when resolving conflicts between aircraft flying 
complex, four-dimensional trajectories. While the 
lowest multiple resolution rate occurred when pilots 
did not follow AOP guidance (Tactical Noncomply), 
this is not seen as a cause for concern. Compliance 
only accounted for 3% of the variance in multiple 
resolutions, and this category of maneuvers had a 
relatively small sample size. In addition, there may 
have been a performance tradeoff, with these 
maneuvers effectively avoiding the intruder aircraft 
at the expense of additional induced conflicts. 

Choice of Maneuver Axis 

To judge the relative effectiveness of lateral and 
vertical AOP guidance, the maneuvers categorized 
above as Strategic Comply and Tactical Comply were 
further separated into Strategic Lateral, Strategic 
Vertical, Tactical Lateral, and Tactical Vertical 
categories. 

Induced Conflicts. Figure 5 shows the percentage of 
induced conflicts that occurred for each of the four 
axis categories. Results from x 2 tests indicate no 
significant differences between either the strategic 
categories (x 2 (l, 77 = 141) = 0.46, p > 0.05) or the 
tactical categories (x 2 (l, N= 118) = 0.37, p > 0.05). 
For the reasons mentioned above, it is not surprising 
that strategic resolutions resulted in fewer induced 
conflicts than tactical resolutions, but within the 
strategic and tactical categories, the choice of 
maneuver axis appears to have had little effect. 



Resolution Type & Axis 

Figure 5. Induced Conflicts vs. Maneuver Axis 
(When AOP Complied With) 

Multiple Resolutions. Figure 6 shows the percentage 
of multiple resolutions that occurred for each of the 
four maneuver axis categories. Results from x 2 tests 
indicate no significant differences between either the 
two strategic categories (x 2 (l, N = 141) = 1.24, p > 
0.05) or the two tactical categories (x 2 (l, N= 118) = 
0.02, p > 0.05). This shows that lateral and vertical 





maneuvers were similarly effective in preventing 
multiple resolutions. However, the slightly higher 
incidence of multiple resolutions for Strategic 
Vertical maneuvers is worth noting. These maneuvers 
required pilots to adjust the autopilot altitude value in 
addition to uploading an FMS route modification. 
There were cases when the altitude value was not 
properly adjusted and the aircraft failed to follow the 
resolution maneuver. Compounding this was the 
difficulty of displaying vertical path changes on a 
horizontal situation display. Ongoing research will 
investigate other options for presenting vertical 
maneuver information, including the use of vertical 
situation displays. 
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Figure 6. Multiple Resolutions vs. Maneuver Axis 
(When AOP Complied With) 

Acceptability of AOP Resolution Guidance 
To examine factors that affect pilot perception of the 
acceptability of AOP resolution guidance, subject 
pilots were asked after each scenario to rate, on a 1 to 
7 scale, the acceptability of a) the first AOP strategic 
resolution in the scenario, and b) AOP tactical 
resolutions in general during the scenario. A series of 
correlations was then performed to determine if 
relationships existed between resolution acceptability 
and four other factors: conflict duration, maneuver 
axis (lateral or vertical), multiple resolutions, and 
induced conflicts. These results are presented in 
Table 3. Overall resolution acceptability was high for 
both strategic resolutions (M= 6.31, SD = 1.28) and 
tactical resolutions (M= 5.12, SD = 1.60). 


Table 3. Resolution Acceptability 


Attribute 

Acceptability Correlation 

Test 

Strategic 
Resolutions 
(N =109) 

Tactical 
Resolutions 
(N= 135) 

Conflict 

Duration 

-0.24* 

0.02 

Pearson 

Maneuver 

Axis 

-0.18 

0.10 

Point- 

biserial 

Multiple 

Resolution 

-0.39* 

0.14 

Point- 

biserial 

Induced 

Conflict 

-0.02 

-0.17* 

Point- 

biserial 


* = significant correlation at p < 0.05 level 


The acceptability of AOP strategic resolution 
maneuvers was significantly correlated with conflict 
duration and multiple resolutions. The significance of 
conflict duration agrees with comments provided 
during debrief sessions, which indicated that pilots 
were frustrated by AOP computation delays and the 
options available when AOP was unable to calculate 
a solution. While the AOP strategic resolution 
algorithm (a genetic algorithm) normally converged 
on a solution within one second, insufficient 
feedback may have been provided to pilots when 
computation times were longer, creating the 
appearance that AOP had “frozen up.” The 
significant correlation with multiple resolutions is 
also reasonable, as one of the primary benefits of 
intent-based, strategic decision support is that the 
necessity for multiple resolution maneuvers should 
be reduced by accounting for trajectory changes that 
would be unknown to a solely state-based system. 

The acceptability of AOP tactical resolutions was 
only significantly correlated with whether or not the 
resolution induced a conflict. As mentioned above, 
depending on the time to predicted LOS, the AOP 
tactical guidance may or may not have accounted for 
aircraft other than the intruder. As such, there were 
cases when the tactical guidance disagreed with 
Maneuver Restriction Bands. Although this behavior 
was explained to subject pilots during training 
exercises, this is recognized as a significant human 
factors issue. Research is underway to modify the 
AOP near-term tactical resolution logic so that 
conflicting information is not presented to pilots. 

Practice Effects 

The En Route Free Maneuvering experiment lasted a 
total of eight days, with three days devoted to 
training, four days for data collection, and one day 
for debriefing. Each data collection day included four 
scenarios, with one at each traffic condition, and with 
the order of conditions varying across days. 

To identify any learning or practice effects, conflicts 
were sorted by day and evaluated with the same 
performance metrics presented above. Figure 7 shows 
the frequency of induced conflicts and multiple 
resolutions across days, x 2 tests indicate that no 
significant differences in the frequency of induced 
conflicts (x 2 (3, N = 332) = 1.37, p > 0.05) or in the 
frequency of multiple resolutions (x 2 (3, N = 332) = 
4.78, p >0.05) existed across the four days. 






Experiment Day 

Figure 7. Resolution Performance by Day 

While no significant practice effects were found, it is 
interesting to compare the performance by day with 
the resolution maneuvers chosen. Figure 8 shows the 
percentage of maneuver types chosen each day. 
Notionally, resolution performance appears to 
degrade with increases in manual and non-complying 
maneuvers over the first three days of the experiment, 
then improve on Day 4 with an increase in Strategic 
Comply maneuvers. 
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Figure 8. Resolution Maneuver Compliance by Day 


feasibility of the En Route Free Maneuvering concept 
while highlighting areas for future research. 
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